Controlling primary and secondary displays from a single touchscreen

ABSTRACT

Various systems and methods for controlling primary and secondary displays from a single touchscreen are described herein. An apparatus comprises a touchscreen display, a processor communicatively coupled to the touchscreen display, a detection module communicatively coupled to the processor, and a control module communicatively coupled to the processor. The detection module detects a particular configuration of a plurality of contact points occurring concurrently on the touchscreen display and detects a touch interaction sequence after detecting the particular configuration of the plurality of contact points. The control module controls a pointer on the secondary display based on the touch interaction sequence.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a U.S. National Stage Application under 35 U.S.C.§371 from International Application No. PCT/US2013/068928, filed Nov. 7,2013, which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

Embodiments described herein generally relate to input devices and inparticular, to a system and method for controlling primary and secondarydisplays from a single touchscreen.

BACKGROUND

In computing, an input device includes any hardware equipment used toprovide data or control signals to a computing device, such as acomputer. Examples of input devices include keyboards, mice, scanners,digital cameras, joysticks, motion detectors, and touchpads.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, which are not necessarily drawn to scale, like numeralsmay describe similar components in different views. Like numerals havingdifferent letter suffixes may represent different instances of similarcomponents. Some embodiments are illustrated by way of example, and notlimitation, in the figures of the accompanying drawings in which:

FIGS. 1A-C illustrate hand positions on a touchscreen, according toexample embodiments;

FIG. 2 is a state diagram illustrating system logic, according to anexample embodiment;

FIG. 3 is a schematic of a computer architecture of atouchscreen-enabled device, according to an example embodiment; and

FIG. 4 is a schematic of a computer architecture of atouchscreen-enabled device, according to an example embodiment;

FIG. 5 is a block diagram illustrating an apparatus to control asecondary display, according to an example embodiment;

FIG. 6 is an example of contact points for a four-finger swipe and amouse-hand configuration for illustration;

FIG. 7 is a flowchart illustrating a method for controlling atouchscreen display and a secondary display from the touchscreendisplay, according to an example embodiment; and

FIG. 8 is a block diagram illustrating an example machine upon which anyone or more of the techniques (e.g., methodologies) discussed herein mayperform, according to an example embodiment.

DETAILED DESCRIPTION

In many mobile devices, conventional input devices such as keyboards andmice have been replaced by a touchscreen interface. Using a touchscreenis intuitive and relatively straightforward. Various gestures have beenimplemented to allow users to interact with the graphical user interfacepresented on a touchscreen interface. Some gestures include selectingitems with a tap or moving items with a tap and drag. Other gesturesinclude multiple fingers (or contact points), such as a two fingerrotate, a two finger zoom, a four finger swipe, or a five finger pinch.These multi-finger gestures perform intuitive actions, such as rotatinga picture with a two finger rotate gesture or magnifying a document viewwith a two finger zoom gesture. Many other gestures may be available toa user for a particular touchscreen interface.

However, when a touchscreen device, such as a tablet computer, is usedwith an additional display device such that the touchscreen's display isextended to the other display device, a problem arises. With thetouchscreen as the only input device available, there is no mechanismimmediately available to control a pointer on the extended display. Whatis needed is an intuitive mechanism to control a pointer on a secondarydisplay from a display of a touchscreen-enabled device. Embodimentsdescribed herein illustrate a mechanism that allows a user to rapidlyswitch between direct manipulation of the touchscreen user interface andcontrol of a pointer on a secondary display without disrupting workflow.In an embodiment, to control the secondary display, a user places herhand on the touchscreen as if she was holding a mouse. This particularhand configuration is identified through the touch contacts with thetouchscreen and the touchscreen-enabled device controls the pointer onthe secondary display using, for example, event messaging.

FIGS. 1A-C illustrate hand positions on a touchscreen, according toexample embodiments. In FIG. 1A, the user is shown interacting with thetouchscreen using one finger (index finger). The single-finger gesturemay be used to select, activate, move, or otherwise manipulate userinterface controls on the touchscreen. In FIG. 1B, the user is shownusing a hand configuration that simulates holding a mouse. In FIG. 1B,four fingers are contacting the touchscreen: the thumb, middle finger,ring finger, and pinky finger. The index finger is elevated in FIG. 1B.While in this position, when the user moves the hand configurationaround the touchscreen while maintaining contact, a pointer is moved ona secondary display—similar to as if the user were using a mouse to movethe pointer. In FIG. 1C, the user's hand is illustrated with the indexfinger down creating a contact point. In this figure, the handconfiguration simulates the position when holding a mouse and clicking amouse button. When used in conjunction with the embodiments describedherein, a mouse down event may be reflected in the secondary display toselect, activate (e.g., execute a program with a double click), or move(e.g., with a click and drag).

While pointing interactions in FIG. 1A on the touchscreen follow an“absolute pointer” paradigm, pointing on the secondary display follows a“relative pointer” paradigm. The pointer interactions on the secondarydisplay may include typical mouse activities, such as moving thepointer, clicking, dragging, or scrolling.

Additionally, when a user runs out of space on the touchscreen displayto move the pointer, the user may pick up her hand and move it to adifferent portion of the touchscreen display and with the correct handconfiguration, continue moving the pointer on the secondary display.This activity is similar to what a user may have to do when running outof room on a desk with a real mouse.

A pointer or mouse cursor is a graphical image presented on a displaydevice that indicates a current point of focus. When combined with aninput action, such as a mouse click or a simulated mouse click, thepointer may be used to select, activate, or move user interface elements(e.g., icons, text, images, cursors, etc.).

When an input is detected, such as a contact with a touchscreen or aclick or motion with a pointing device (e.g., a mouse), an event isgenerated. The event may be used to initiate other actions or processes.Event-driven programming and environments use events to controlprogrammatic responses. In this document events are described withrespect to a contact or set of contacts with a touchscreen input device.The contact or set of contacts may be interpreted as a gesture. Thecontact may be referred to by an event name or category, such as mousedown (e.g., contact initiated), mouse up (e.g., contact terminated),mouse move, or the like. Although some examples may use terminology inthe context of mouse events, it is understood that any type of eventcaused by an actuation of an input device may be used.

Using a gesture as illustrated in FIGS. 1A-C ensures that the user isallowed to transition between input modes quickly and intuitively. Asdiscussed above, some touchscreen devices allow for four and five-fingergestures. In these situations, a timeout may be used to distinguish themouse-hand configuration from another four-finger gesture. For example,a user may rest their hand in a mouse-hand configuration in a relativelystill position for more than a threshold period (e.g., 0.5 seconds) inorder to activate the pointer control mode. In this manner, thefour-finger mouse-hand gesture may be distinguished from a four-fingerswipe to change applications, for example.

In an embodiment, an audible, visual, or tactile cue may be used toinform the user that the control has switched from direct input on thetouchscreen to controlling a pointer on a secondary display. Forexample, a sound may be played during a control change, the pointer maybe highlighted on the secondary display, or the touchscreen device mayvibrate to indicate a control change. Combinations of audible, visual,or tactile cues may be used. Additionally, the cue or cues may beconfigurable by a user (e.g., selecting the alert sound or volume,selecting which cues are used when, or selecting duration of a cue).

FIG. 2 is a state diagram 200 illustrating system logic, according to anexample embodiment. The initial default state (block 202) is with nofingers touching the touchscreen. This state is entered whenever theuser is not touching the screen. In an embodiment, it is the only statethat allows the user to transition between interacting with elements onthe touchscreen and interacting with elements on the secondary display.

When a user touches the touchscreen with one to three fingers, the 1-3finger gestures are processed (block 204). Examples of 1-3 fingergestures include, but are not limited to tapping, tap and hold, tap anddrag, pinch, rotate, pan, scroll, or swipe. Such gestures are processedas taps, movements, or combinations of taps and movements. The gestureprocessing ends when all fingers are lifted from the touchscreen (returnto default state 202). If during processing of a 1-3 finger gesture(block 204), additional contact points are made, they are ignored. Forexample, if the user touches the screen with additional fingers (e.g.,fingers four or five from one hand), then no events are delivered untileither the user resumes a 1-3 finger touch gesture or the user lifts allfingers (0 finger touch). When additional fingers greater than the 1-3fingers used in a gesture are detected, the state diagram 200 enters asink 206 where no touch commands are processed. Alternatively, whenextra touches are detected, a previously-initiated gesture may continue.For example, if a user is zooming/shrinking with a two-finger touchpinch gesture and accidentally touches the screen with a third or fourthfinger, the zooming/shrinking gesture may continue to operate on theinitial two touch points and ignore the additional touch points.

From the default state 202, if the user touches the touchscreen withfour or five fingers, then a timeout/trigger (block 208) is optionallyused to determine the intent of the user. The timeout/trigger 208 may beused in systems where a four or five-finger touch gesture is available.For example, when a system provides for a four-finger swipe gesture toswitch applications, this four-finger gesture needs to be distinguishedfrom the four-finger gesture designed to operate a secondary display.Thus, in an embodiment, a timeout/trigger determination is used. Thetimeout may be a delay period, such as 0.5 seconds, where the userplaces four fingers on the touchscreen and remains relatively stationaryfor the delay period. Using a timeout, if the user wishes to changeapplications, a four-finger gesture of placing the fingers on thetouchscreen and then swiping relatively immediately afterwards isdistinguishable from a gesture used to control a separate display.

In another embodiment, a trigger determination may be used. The triggermay be a certain arrangement, orientation, or configuration of contactpoints, such as in a position that would emulate holding a mouse. Usingthis particular finger/hand position, the user's intent may be inferredand the control of the secondary display may be initiated. Theparticular finger/hand configuration may be configured by the user. Forexample, the user may cause the touchscreen device to enter a learningmode and then place their hand in a mouse-hand configuration. Thetouchscreen device may then associate the user's touch points with amouse-hand configuration such that at a later time, when the user placesher hand in the pre-determined position, the mode for pointer control ona secondary display is initiated.

If the touchscreen-enabled device does not support four or five fingergestures, then the timeout/trigger determination 202 may be unnecessaryand may not be used. However, in some cases, the timeout/triggermechanism may still be used even when other four or five finger gesturesare not supported in order to maintain a consistent user interface andimplementation between various platforms.

When the user touches the touchscreen in a way to pass thetimeout/trigger determination (if there is one), then the touchinteraction sequence is directed to the secondary display. Initially,the user may be in the four-fingers contact state (block 210). If theuser moves their hand on the screen then a move event is directed to thesecondary display to move the pointer. If the user puts a fifth fingeron the touchscreen, such as an index finger, then the logic flows to thefive-fingers contact state (block 212). Lifting the fifth finger willreturn the process to the four-finger contact state (block 210).Touching and lifting the fifth finger (in quick succession) may resultin a tap interaction sequence, whereas adding the fifth finger (e.g.,tap and hold) may result in a tap and drag interaction sequence. Othergestures may be used in a four and five-finger interaction sequence. Forexample, the tap and drag with an index finger is similar to a motionused to actuate a scroll wheel on a physical mouse. In an embodiment, atap and drag with the fifth finger may result in a scrolling gesture.

In the five-finger contact state (block 212), when a user moves thecontact points, then a move event may be directed to the secondarydisplay. In an example, the five-finger move is considered a select andmove sequence (e.g., to select a file and drag it from one folder toanother folder).

In either the four-finger contact state (block 210) of the five-fingercontact state (block 212), lifting fingers may result in the logicmoving to a sink state (block 214A, 214B), where touch processing issuspended. Alternatively, lifting of fingers may be ignored and theprevious gesture processing may be continued (where at least one fingerremains in contact). For example, when a user has four fingers incontact and is moving the pointer around on the secondary display, ifthe user lifts a finger or two from contact, the process flow maycontinue to move the pointer as if the user still had four fingers incontact.

From either the four-finger contact state (block 210) of the five-fingercontact state (block 212), when the user removes all fingers fromcontact with the touchscreen, the process flow returns to the initialdefault state (block 202).

It is understood that various event messages may be initiated orcommunicated at various points in the state diagram 200. For example,when moving from the initial default state (block 202) to the 1-3fingers down state (block 204), an operating system level event messagemay be generated to indicate the start of a touch interaction sequence.Similarly, as a user moves their finger(s) while interacting with thetouchscreen, various event messages may be generated to indicate thatthe touches moved and in what manner (direction, velocity, pressure,etc.). When operating to interact with the secondary display, mouseevents may be generated and transmitted to an application controllingthe pointer on the secondary display.

FIG. 3 is a schematic of a computer architecture of atouchscreen-enabled device 300, according to an example embodiment. Thetouchscreen-enabled device 300 includes a touchscreen 302, a displaydriver 304, and an operating system 306. The operating system 306receives input and provides output via the touchscreen 302. The displaydriver 304 operates with the operating system 306 to produce the displayon the touchscreen 302. The display driver 304 has access to memory 308.The memory 308 may be system memory (e.g., random access memory (RAM))or dedicated memory, such as video memory (e.g., VRAM). Thetouchscreen-enabled device 300 is connected to a secondary display 310over a connection 312. The connection 312 may be any kind of connectionincluding physical or wireless connections. In various embodiments, theconnection 312 includes a wireless network including, but not limited towireless display (WiDi) technology, Bluetooth®, IEEE 802.11, Wi-Fi™, orother personal area networks or wireless local area networks. In otherembodiments, the connection 312 includes a universal serial bus (USB),high-definition multimedia interface (HDMI), video graphics array (VGA),or digital visual interface (DVI) connection.

In the embodiment illustrated in FIG. 3, the secondary display 310 maybe memory mapped to the memory 308 such that the display driver 304 isable to update and manage the presentation on the secondary display 310.In order to present the image, the touchscreen-enabled device 300transmits a video signal to the secondary display 310 over theconnection 312. In this configuration, an application 312 executing onthe touchscreen-enabled device 300 is displayed on the secondary display310 (e.g., extended display) where pointer input to the application 312being displayed on the secondary display 310 is implemented with aspecial gesture (e.g., a mouse-hand configuration). The application 312may be programmed to detect a mouse-like event, such as that generatedwhen the user interacts with the touchscreen-enabled device 300 with amouse-hand configuration. The application 312 may then respond to themouse-like event, such as by way of activating a user interface elementor executing a process or subprocess. The application 312 in conjunctionwith the operating system 306 and display driver 304 may control theoutput presentation of the pointer on the secondary display 310. Forother applications that are presented on the touchscreen 302, the usermay use typical gesture input.

FIG. 4 is a schematic of a computer architecture of atouchscreen-enabled device 400, according to an example embodiment.Similar to the configuration in FIG. 3, the touchscreen-enabled device400 includes a touchscreen 402, a display driver 404, an operatingsystem 406, and a memory 408. However, in FIG. 4, the secondary display410 is part of a separate computing system 414 with an operating system416 and a display driver 418. In this configuration, the operatingsystem 416 and display driver 418 are used to present information on thesecondary display 410. In particular, the operating system 416 anddisplay driver 418 are used to display and update a pointer on thesecondary display 410. Thus, in the embodiment illustrated in FIG. 4,when a user operates the touchscreen-enabled device 400 to move thepointer on the secondary display 410, instead of transmitting a videosignal to the secondary display 410 via the connection 412, eventmessages are transmitted. For example, a standardized event message forvarious mouse movements or actions may be transmitted from thetouchscreen-enabled device 400 to the operating system 416 of thesecondary display 410. Examples may be a mouse_down, mouse_up,mouse_move, mouse_scroll, or mouse_click event or other types of events.Event names and parameters may differ based on the operating system.

Instead of transmitting the event messages to the operating system 416,an application 420 may be installed on the computing system. Theapplication 420 may receive mouse event messages and update the pointerposition or actions based on the messages. The application 420 may beany of a variety of applications, such as a multimedia presentationapplication (e.g., movie player or slideshow player), an entertainmentapplication (e.g., a game), or a utility application (e.g., a wordprocessor, email, or office work application).

Thus, in the embodiment illustrated in FIG. 4, a user is able to controlapplications presented on the touchscreen-enabled device 400 withtypical gestures and then control applications presented on thesecondary display 410 with special gestures (e.g., a mouse-handconfiguration). Switching control between the two is quick, easy, andintuitive.

FIG. 5 is a block diagram illustrating an apparatus 500 to control asecondary display 502, according to an example embodiment. The apparatus500 includes a touchscreen display 504, a processor 506, a detectionmodule 508, and a control module 510. The processor 506, detectionmodule 508, and control module 510 are communicatively coupled to oneanother. The processor 506 is communicatively coupled to the touchscreendisplay 504.

In an embodiment, the detection module detects a particularconfiguration of a plurality of contact points occurring concurrently onthe touchscreen display and detects a touch interaction sequence afterdetecting the particular configuration of the plurality of contactpoints. A touch interaction sequence is a series of one or moreconcurrent touches and subsequent actions. The subsequent actions may bea drag (resulting in a touch-and-drag sequence), a hold (resulting in atouch-and-hold sequence), a tap (resulting in a single, double, or othersequence of taps), or the like. Touch-and-drag sequences may be used tomagnify or shrink (e.g., a two-finger touch-and-drag sequence ofpinching), rotate an image, or change applications. Touch-and-holdinteraction sequences may be used to select or activate an object in auser interface, such as to cause a contextual menu or submenu to appear,select a folder or document icon, or activate a user interface element.

In an embodiment, the particular configuration of the plurality ofcontact points form a mouse-hand configuration. A mouse-handconfiguration is an arrangement of contact points that resemble whatwould exist if a person where holding or handling a mouse input device.This may be distinguished from a four-finger swipe, which may beperformed usually with the index, middle, ring, and pinky fingers,because a four-finger swipe contact pattern may be roughly linear orwith a slight arc. In contrast, a mouse-hand configuration may havethree contact points that are in a rough linear pattern or with a slightarc, and a fourth contact point that is separate in distance andalignment from the first three contact points. FIG. 6 is an example ofcontact points for a four-finger swipe 600 and a mouse-handconfiguration 602 for illustration. As illustrated in FIG. 6, in anembodiment, the plurality of contact points for a mouse-handconfiguration 602 is four contact points (with a fifth contact pointoptional). Other numbers of contact points may be used in the particularconfiguration, such as three. Additionally, a palm contact point may beused as a portion of the particular configuration (e.g., where a userrests her palm or a portion of the heel of the hand on the touchscreenwhile “mousing”).

The control module controls a pointer on the secondary display based onthe touch interaction sequence, the secondary display communicativelycoupled to the apparatus.

In an embodiment, the apparatus 500 includes a configuration module 512communicatively coupled to the processor, where the configuration module512 enters a learning mode and configures the particular configurationof the plurality of contact points based on a user's hand positionduring the learning mode. For example, the user may use a configurationscreen or other mechanism to assist the apparatus 500 in recognizing amouse-hand configuration of the user. This may be useful especially whena user is left handed. The configuration process may include having theuser place her hand on the touchscreen one or several times to identifythe approximate pattern and alignment of contact points for the user'smouse-hand configuration.

The configuration module 512 may allow a user to configure the timeoutdelay before entering a control mode to control a secondary display. Thetimeout delay may be relatively short periods of time, such as 0.1, 0.2,or 0.5 seconds. The configuration module 512 may also be adaptable suchthat as the user uses the control mode to control the secondary display,the configuration module 512 is able to more quickly recognize theuser's hand position and may then reduce or eliminate the timeout periodbefore entering the control mode to control the secondary display.

In an embodiment, the detection module detects the touch interactionsequence by detecting a drag movement of the plurality of contactpoints. In such an embodiment, the control module 510 controls thepointer on the secondary display by moving a pointer on the secondarydisplay. This may be performed by updating a pointer position in memoryand sending a video signal to the secondary display 502 with the pointerin the new position. Alternatively, a mouse event may be transmitted toan application or an operating system that controls the presentation onthe secondary display 502. Thus, in a further embodiment, moving thepointer on the secondary display includes transmitting a mouse moveevent to the secondary display 502.

In an embodiment, the detection module detects the touch interactionsequence by: detecting a new contact point in addition to the pluralityof contact points and detecting a tapping motion at the new contactpoint. In such an embodiment, the control module 510 controls thepointer on the secondary display by selecting an object under thepointer at the secondary display. In a further embodiment, selecting theobject under the pointer at the secondary display includes transmittinga mouse down event to the secondary display and transmitting a mouse upevent to the secondary display.

In an embodiment, the detection module detects the touch interactionsequence by detecting a new contact point in addition to the pluralityof contact points and detecting a swiping motion originating from thenew contact point. In such an embodiment, the control module 510controls the pointer on the secondary display includes scrolling awindow in the secondary display. In a further embodiment, scrolling thewindow in the secondary display includes transmitting a mouse scrollevent to the secondary display.

As described in FIGS. 3 and 4, the apparatus 500 and secondary display502 may be communicatively connected using various physical or wirelessconnection mediums and protocols, such as HDMI, USB, Wi-Fi, or WiDi.Thus, in an embodiment, the apparatus 500 and the secondary display 502are communicatively coupled over a wireless network connection. In afurther embodiment, the wireless network connection is a wirelessdisplay connection.

FIG. 7 is a flowchart illustrating a method 700 for controlling atouchscreen display and a secondary display from the touchscreendisplay, according to an example embodiment. At block 702, a particularconfiguration of a plurality of contact points occurring concurrently onthe touchscreen display is detected. At block 704, a touch interactionsequence is detected after detecting the particular configuration of theplurality of contact points. At block 706, a pointer is controlled onthe secondary display based on the touch interaction sequence. In anembodiment, the particular configuration of the plurality of contactpoints form a mouse-hand configuration. In an embodiment, the pluralityof contact points is four contact points.

In an embodiment, the method 700 includes entering a learning mode andconfiguring the particular configuration of the plurality of contactpoints based on a user's hand position during the learning mode.

In an embodiment, detecting the touch interaction sequence includesdetecting a drag movement of the plurality of contact points andcontrolling the pointer on the secondary display includes moving apointer on the secondary display. In a further embodiment, moving thepointer on the secondary display includes transmitting a mouse moveevent to the secondary display.

In an embodiment, detecting the touch interaction sequence includes:detecting a new contact point in addition to the plurality of contactpoints and detecting a tapping motion at the new contact point. In suchan embodiment, controlling the pointer on the secondary display includesselecting an object under the pointer at the secondary display. In afurther embodiment, selecting the object under the pointer at thesecondary display includes transmitting a mouse down event to thesecondary display and transmitting a mouse up event to the secondarydisplay.

In an embodiment, detecting the touch interaction sequence includesdetecting a new contact point in addition to the plurality of contactpoints and detecting a swiping motion originating from the new contactpoint. In such an embodiment, controlling the pointer on the secondarydisplay includes scrolling a window in the secondary display. In afurther embodiment, scrolling the window in the secondary displayincludes transmitting a mouse scroll event to the secondary display.

Embodiments may be implemented in one or a combination of hardware,firmware, and software. Embodiments may also be implemented asinstructions stored on a machine-readable storage device, which may beread and executed by at least one processor to perform the operationsdescribed herein. A machine-readable storage device may include anynon-transitory mechanism for storing information in a form readable by amachine (e.g., a computer). For example, a machine-readable storagedevice may include read-only memory (ROM), random-access memory (RAM),magnetic disk storage media, optical storage media, flash-memorydevices, and other storage devices and media.

Examples, as described herein, may include, or may operate on, logic ora number of components, modules, or mechanisms. Modules may be hardware,software, or firmware communicatively coupled to one or more processorsin order to carry out the operations described herein. Modules mayhardware modules, and as such modules may be considered tangibleentities capable of performing specified operations and may beconfigured or arranged in a certain manner. In an example, circuits maybe arranged (e.g., internally or with respect to external entities suchas other circuits) in a specified manner as a module. In an example, thewhole or part of one or more computer systems (e.g., a standalone,client or server computer system) or one or more hardware processors maybe configured by firmware or software (e.g., instructions, anapplication portion, or an application) as a module that operates toperform specified operations. In an example, the software may reside ona machine-readable medium. In an example, the software, when executed bythe underlying hardware of the module, causes the hardware to performthe specified operations. Accordingly, the term hardware module isunderstood to encompass a tangible entity, be that an entity that isphysically constructed, specifically configured (e.g., hardwired), ortemporarily (e.g., transitorily) configured (e.g., programmed) tooperate in a specified manner or to perform part or all of any operationdescribed herein. Considering examples in which modules are temporarilyconfigured, each of the modules need not be instantiated at any onemoment in time. For example, where the modules comprise ageneral-purpose hardware processor configured using software; thegeneral-purpose hardware processor may be configured as respectivedifferent modules at different times. Software may accordingly configurea hardware processor, for example, to constitute a particular module atone instance of time and to constitute a different module at a differentinstance of time. Modules may also be software or firmware modules,which operate to perform the methodologies described herein.

FIG. 8 is a block diagram illustrating a machine in the example form ofa computer system 800, within which a set or sequence of instructionsmay be executed to cause the machine to perform any one of themethodologies discussed herein, according to an example embodiment. Inalternative embodiments, the machine operates as a standalone device ormay be connected (e.g., networked) to other machines. In a networkeddeployment, the machine may operate in the capacity of either a serveror a client machine in server-client network environments, or it may actas a peer machine in peer-to-peer (or distributed) network environments.The machine may be an onboard vehicle system, personal computer (PC), atablet PC, a hybrid tablet, a personal digital assistant (PDA), a mobiletelephone, or any machine capable of executing instructions (sequentialor otherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein. Similarly, the term“processor-based system” shall be taken to include any set of one ormore machines that are controlled by or operated by a processor (e.g., acomputer) to individually or jointly execute instructions to perform anyone or more of the methodologies discussed herein.

Example computer system 800 includes at least one processor 802 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) or both,processor cores, compute nodes, etc.), a main memory 804 and a staticmemory 806, which communicate with each other via a link 808 (e.g.,bus). The computer system 800 may further include a video display unit810, an alphanumeric input device 812 (e.g., a keyboard), and a userinterface (UI) navigation device 814 (e.g., a mouse). In one embodiment,the video display unit 810, input device 812 and UI navigation device814 are incorporated into a touch screen display. The computer system800 may additionally include a storage device 816 (e.g., a drive unit),a signal generation device 818 (e.g., a speaker), a network interfacedevice 820, and one or more sensors (not shown), such as a globalpositioning system (GPS) sensor, compass, accelerometer, or othersensor.

The storage device 816 includes a machine-readable medium 822 on whichis stored one or more sets of data structures and instructions 824(e.g., software) embodying or utilized by any one or more of themethodologies or functions described herein. The instructions 824 mayalso reside, completely or at least partially, within the main memory804, static memory 806, and/or within the processor 802 during executionthereof by the computer system 800, with the main memory 804, staticmemory 806, and the processor 802 also constituting machine-readablemedia.

While the machine-readable medium 822 is illustrated in an exampleembodiment to be a single medium, the term “machine-readable medium” mayinclude a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more instructions 824. The term “machine-readable medium”shall also be taken to include any tangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machineand that cause the machine to perform any one or more of themethodologies of the present disclosure or that is capable of storing,encoding or carrying data structures utilized by or associated with suchinstructions. The term “machine-readable medium” shall accordingly betaken to include, but not be limited to, solid-state memories, andoptical and magnetic media. Specific examples of machine-readable mediainclude non-volatile memory, including but not limited to, by way ofexample, semiconductor memory devices (e.g., electrically programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM)) and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks.

The instructions 824 may further be transmitted or received over acommunications network 826 using a transmission medium via the networkinterface device 820 utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, plain old telephone (POTS)networks, and wireless data networks (e.g., Wi-Fi, 3G, and 4G LTE/LTE-Aor WiMAX networks). The term “transmission medium” shall be taken toinclude any intangible medium that is capable of storing, encoding, orcarrying instructions for execution by the machine, and includes digitalor analog communications signals or other intangible medium tofacilitate communication of such software.

Example 1 includes subject matter (such as a device, apparatus, ormachine) comprising an apparatus to control primary and secondarydisplays from a single touchscreen, comprising: a touchscreen display; aprocessor communicatively coupled to the touchscreen display; adetection module communicatively coupled to the processor, the detectionmodule to: detect a particular configuration of a plurality of contactpoints occurring concurrently on the touchscreen display; and detect atouch interaction sequence after detecting the particular configurationof the plurality of contact points; and a control module communicativelycoupled to the processor, the control module to control a pointer on thesecondary display based on the touch interaction sequence, the secondarydisplay communicatively coupled to the apparatus.

Example 2, the subject matter of Example 1 may optionally include, aconfiguration module communicatively coupled to the processor, theconfiguration module to: enter a learning mode; and configure theparticular configuration of the plurality of contact points based on auser's hand position during the learning mode.

In Example 3 the subject matter of any one or more of Examples 1 to 2may optionally include, wherein the particular configuration of theplurality of contact points form a mouse-hand configuration.

In Example 4 the subject matter of any one or more of Examples 1 to 3may optionally include, wherein the plurality of contact points is fourcontact points.

In Example 5 the subject matter of any one or more of Examples 1 to 4may optionally include, wherein the detection module detects the touchinteraction sequence by detecting a drag movement of the plurality ofcontact points; and wherein controlling the pointer on the secondarydisplay includes moving a pointer on the secondary display.

In Example 6 the subject matter of any one or more of Examples 1 to 5may optionally include, wherein detecting the drag movement includesdetecting a mouse move event.

In Example 7 the subject matter of any one or more of Examples 1 to 6may optionally include, wherein moving the pointer on the secondarydisplay includes transmitting a mouse move event to the secondarydisplay.

In Example 8 the subject matter of any one or more of Examples 1 to 7may optionally include, wherein the detection module detects the touchinteraction sequence by: detecting a new contact point in addition tothe plurality of contact points; and detecting a tapping motion at thenew contact point; and wherein controlling the pointer on the secondarydisplay includes: selecting an object under the pointer at the secondarydisplay.

In Example 9 the subject matter of any one or more of Examples 1 to 8may optionally include, wherein detecting the tapping motion includesdetecting a mouse down and mouse up event sequence.

In Example 10 the subject matter of any one or more of Examples 1 to 9may optionally include, wherein selecting the object under the pointerat the secondary display includes: transmitting a mouse down event tothe secondary display; and transmitting a mouse up event to thesecondary display.

In Example 11 the subject matter of any one or more of Examples 1 to 10may optionally include, wherein the detection module detects the touchinteraction sequence by: detecting a new contact point in addition tothe plurality of contact points; and detecting a swiping motionoriginating from the new contact point; and wherein controlling thepointer on the secondary display includes scrolling a window in thesecondary display.

In Example 12 the subject matter of any one or more of Examples 1 to 11may optionally include, wherein detecting the swiping motion includesdetecting a mouse scroll event.

In Example 13 the subject matter of any one or more of Examples 1 to 12may optionally include, wherein scrolling the window in the secondarydisplay includes transmitting a mouse scroll event to the secondarydisplay.

In Example 14 the subject matter of any one or more of Examples 1 to 13may optionally include, wherein the apparatus and the secondary displayare communicatively coupled over a wireless network connection.

In Example 15 the subject matter of any one or more of Examples 1 to 14may optionally include, wherein the wireless network connection is awireless display connection.

Example 16 includes or may optionally be combined with the subjectmatter of any one of Examples 1-15 to include subject matter forcontrolling a touchscreen display and a secondary display from thetouchscreen display (such as a method, means for performing acts,machine readable medium including instructions that when performed by amachine cause the machine to performs acts, or an apparatus configuredto perform) comprising: detecting a particular configuration of aplurality of contact points occurring concurrently on the touchscreendisplay; detecting a touch interaction sequence after detecting theparticular configuration of the plurality of contact points; andcontrolling a pointer on the secondary display based on the touchinteraction sequence.

In Example 17, the subject matter of Example 16 may optionally include,entering a learning mode; and configuring the particular configurationof the plurality of contact points based on a user's hand positionduring the learning mode.

In Example 18 the subject matter of any one or more of Examples 16 to 17may optionally include, wherein the particular configuration of theplurality of contact points form a mouse-hand configuration.

In Example 19 the subject matter of any one or more of Examples 16 to 18may optionally include, wherein the plurality of contact points is fourcontact points.

In Example 20 the subject matter of any one or more of Examples 16 to 19may optionally include, wherein detecting the touch interaction sequenceincludes detecting a drag movement of the plurality of contact points;and wherein controlling the pointer on the secondary display includesmoving a pointer on the secondary display.

In Example 21 the subject matter of any one or more of Examples 16 to 20may optionally include, wherein detecting the drag movement includesdetecting a mouse move event.

In Example 22 the subject matter of any one or more of Examples 16 to 21may optionally include, wherein moving the pointer on the secondarydisplay includes transmitting a mouse move event to the secondarydisplay.

In Example 23 the subject matter of any one or more of Examples 16 to 22may optionally include, wherein detecting the touch interaction sequenceincludes: detecting a new contact point in addition to the plurality ofcontact points; and detecting a tapping motion at the new contact point;and wherein controlling the pointer on the secondary display includes:selecting an object under the pointer at the secondary display.

In Example 24 the subject matter of any one or more of Examples 16 to 23may optionally include, wherein detecting the tapping motion includesdetecting a mouse down and mouse up event sequence.

In Example 25 the subject matter of any one or more of Examples 16 to 24may optionally include, wherein selecting the object under the pointerat the secondary display includes: transmitting a mouse down event tothe secondary display; and transmitting a mouse up event to thesecondary display.

In Example 26 the subject matter of any one or more of Examples 16 to 25may optionally include, wherein detecting the touch interaction sequenceincludes: detecting a new contact point in addition to the plurality ofcontact points; and detecting a swiping motion originating from the newcontact point; and wherein controlling the pointer on the secondarydisplay includes scrolling a window in the secondary display.

In Example 27 the subject matter of any one or more of Examples 16 to 26may optionally include, wherein detecting the swiping motion includesdetecting a mouse scroll event.

In Example 28 the subject matter of any one or more of Examples 16 to 27may optionally include, wherein scrolling the window in the secondarydisplay includes transmitting a mouse scroll event to the secondarydisplay.

Example 29 includes or may optionally be combined with the subjectmatter of any one of Examples 1-28 to include a machine-readable mediumincluding instructions for anomaly detection on a web client, which whenexecuted by a machine, cause the machine to perform operations of anyone of the examples 1-28.

Example 30 includes or may optionally be combined with the subjectmatter of any one of Examples 1-28 to include an apparatus comprisingmeans for performing any of the examples 1-28.

Example 31 includes an apparatus for controlling a touchscreen displayand a secondary display from the touchscreen display, the apparatuscomprising: means for detecting a particular configuration of aplurality of contact points occurring concurrently on the touchscreendisplay; means for detecting a touch interaction sequence afterdetecting the particular configuration of the plurality of contactpoints; and means for controlling a pointer on the secondary displaybased on the touch interaction sequence.

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments that may bepracticed. These embodiments are also referred to herein as “examples.”Such examples may include elements in addition to those shown ordescribed. However, also contemplated are examples that include theelements shown or described. Moreover, also contemplate are examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

Publications, patents, and patent documents referred to in this documentare incorporated by reference herein in their entirety, as thoughindividually incorporated by reference. In the event of inconsistentusages between this document and those documents so incorporated byreference, the usage in the incorporated reference(s) are supplementaryto that of this document; for irreconcilable inconsistencies, the usagein this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In the appended claims, the terms “including” and“in which” are used as the plain-English equivalents of the respectiveterms “comprising” and “wherein.” Also, in the following claims, theterms “including” and “comprising” are open-ended, that is, a system,device, article, or process that includes elements in addition to thoselisted after such a term in a claim are still deemed to fall within thescope of that claim. Moreover, in the following claims, the terms“first,” “second,” and “third,” etc. are used merely as labels, and arenot intended to suggest a numerical order for their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with others. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is to allow thereader to quickly ascertain the nature of the technical disclosure, forexample, to comply with 37 C.F.R. §1.72(b) in the United States ofAmerica. It is submitted with the understanding that it will not be usedto interpret or limit the scope or meaning of the claims. Also, in theabove Detailed Description, various features may be grouped together tostreamline the disclosure. However, the claims may not set forth everyfeature disclosed herein as embodiments may feature a subset of saidfeatures. Further, embodiments may include fewer features than thosedisclosed in a particular example. Thus, the following claims are herebyincorporated into the Detailed Description, with a claim standing on itsown as a separate embodiment. The scope of the embodiments disclosedherein is to be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled.

What is claimed is:
 1. An apparatus to control a secondary display, theapparatus comprising: a touchscreen display; a processor communicativelycoupled to the touchscreen display; a detection module communicativelycoupled to the processor, the detection module to: detect a particularconfiguration of a plurality of contact points occurring concurrently onthe touchscreen display, the particular configuration forming amouse-hand configuration of a single hand having contact pointsrepresenting contact with a thumb and at least two additional fingers ofthe same hand; and detect a touch interaction sequence after detectingthe particular configuration of the plurality of contact points, whereinthe touch interaction sequence includes simultaneous motion of theplurality of contact points while maintaining the particularconfiguration; a control module communicatively coupled to theprocessor, the control module to switch from a direct-control mode ofcontrolling of the apparatus via the touchscreen display, to aremote-control mode of controlling a pointer on the secondary displaybased on the touch interaction sequence; and a configuration modulecommunicatively coupled to the processor, the configuration module to:enter a learning mode; capture a user's hand position during thelearning mode, the hand position engaging the plurality of contactpoints occurring concurrently on the touchscreen display; and configurethe particular configuration of the plurality of contact points based onthe user's hand position during the learning mode.
 2. The apparatus ofclaim 1, wherein the control module is to control the pointer on thesecondary display by at least transmitting a mouse move event to thesecondary display.
 3. The apparatus of claim 1, wherein the detectionmodule detects the touch interaction sequence by: detecting a newcontact point in addition to the plurality of contact points; anddetecting a tapping motion at the new contact point; and whereincontrolling the pointer on the secondary display includes: selecting anobject under the pointer at the secondary display.
 4. The apparatus ofclaim 3, wherein selecting the object under the pointer at the secondarydisplay includes: transmitting a mouse down event to the secondarydisplay; and transmitting a mouse up event to the secondary display. 5.The apparatus of claim 1, wherein the detection module detects the touchinteraction sequence by: detecting a new contact point in addition tothe plurality of contact points; and detecting a swiping motionoriginating from the new contact point; and wherein controlling thepointer on the secondary display includes scrolling a window in thesecondary display.
 6. The apparatus of claim 5, wherein scrolling thewindow in the secondary display includes transmitting a mouse scrollevent to the secondary display.
 7. The apparatus of claim 1, wherein theapparatus and the secondary display are communicatively coupled over awireless network connection.
 8. The apparatus of claim 7, wherein thewireless network connection is a wireless display connection.
 9. Anon-transitory machine-readable medium including instructions forcontrolling a touchscreen display and a secondary display from thetouchscreen display, which when executed by a machine, cause the machineto perform operations comprising: detecting a particular configurationof a plurality of contact points occurring concurrently on thetouchscreen display, the particular configuration forming a mouse-handconfiguration of a single hand having contact points representingcontact with a thumb and at least two additional fingers of the samehand; detecting a touch interaction sequence after detecting theparticular configuration of the plurality of contact points, wherein thetouch interaction sequence includes simultaneous motion of the pluralityof contact points while maintaining the particular configuration; andswitching from a direct-control mode of controlling of the machine viathe touchscreen display, to a remote-control mode of controlling apointer on the secondary display based on the touch interactionsequence; entering a learning mode; capturing a user's hand positionduring the learning mode, the hand position engaging the plurality ofcontact points occurring concurrently on the touchscreen display; andconfiguring the particular configuration of the plurality of contactpoints based on a user's hand position during the learning mode.
 10. Thenon-transitory machine-readable medium of claim 9, wherein controllingthe pointer on the secondary display includes transmitting a mouse moveevent to the secondary display.
 11. The non-transitory machine-readablemedium of claim 9, wherein detecting the touch interaction sequenceincludes: detecting a new contact point in addition to the plurality ofcontact points; and detecting a tapping motion at the new contact point;and wherein controlling the pointer on the secondary display includes:selecting an object under the pointer at the secondary display.
 12. Thenon-transitory machine-readable medium of claim 11, wherein selectingthe object under the pointer at the secondary display includes:transmitting a mouse down event to the secondary display; andtransmitting a mouse up event to the secondary display.
 13. Thenon-transitory machine-readable medium of claim 9, wherein detecting thetouch interaction sequence includes: detecting a new contact point inaddition to the plurality of contact points; and detecting a swipingmotion originating from the new contact point; and wherein controllingthe pointer on the secondary display includes scrolling a window in thesecondary display.
 14. A method for controlling a touchscreen displayand a secondary display from the touchscreen display, the methodcomprising: detecting a particular configuration of a plurality ofcontact points occurring concurrently on the touchscreen display, theparticular configuration forming a mouse-hand configuration of a singlehand having contact points representing contact with a thumb and atleast two additional fingers of the same hand; detecting a touchinteraction sequence after detecting the particular configuration of theplurality of contact points, wherein the touch interaction sequenceincludes simultaneous motion of the plurality of contact points whilemaintaining the particular configuration; and switching from adirect-control mode of controlling of a device incorporating thetouchscreen display, to a remote-control mode of controlling a pointeron the secondary display based on the touch interaction sequence;entering a learning mode; capturing a user's hand position during thelearning mode, the hand position engaging the plurality of contactpoints occurring concurrently on the touchscreen display; andconfiguring the particular configuration of the plurality of contactpoints based on a user's hand position during the learning mode.
 15. Themethod of claim 14, wherein detecting the touch interaction sequenceincludes: detecting a new contact point in addition to the plurality ofcontact points; and detecting a tapping motion at the new contact point;and wherein controlling the pointer on the secondary display includes:selecting an object under the pointer at the secondary display.